@dataknut)If you wish to refer to any of the material from this report please cite as:
Report circulation:
This work is (c) 2020 the University of Southampton.
sotonAirDT[, `:=`(obsDate, lubridate::date(dateTimeUTC))]
sotonAirDT[, `:=`(year, lubridate::year(dateTimeUTC))]
sotonAirDT[, `:=`(origDoW, lubridate::wday(dateTimeUTC, label = TRUE))]
sotonAirDT[, `:=`(month, lubridate::month(obsDate))]
sotonAirDT[, `:=`(site, ifelse(site == "Southampton A33", "Southampton A33\n(via AURN)", site))]
sotonAirDT[, `:=`(site, ifelse(site == "Southampton Centre", "Southampton Centre\n(via AURN)", site))]
extractDT <- sotonAirDT[!is.na(value)] # leave out 2016 so we compare with previous 3 years
# this is such a kludge
extractDT[, `:=`(decimalDate, lubridate::decimal_date(obsDate))] # gives year.% of year
# set to 2020 'dates'
extractDT[, `:=`(date2020, lubridate::as_date(lubridate::date_decimal(2020 + (decimalDate - year))))] # sets 'year' portion to 2020 so the lockdown annotation works
extractDT[, `:=`(day2020, lubridate::wday(date2020, label = TRUE))] #
# 2020 Jan 1st = Weds
dt2020 <- extractDT[year == 2020]
dt2020[, `:=`(fixedDate, obsDate)] # no need to change
dt2020[, `:=`(fixedDoW, lubridate::wday(fixedDate, label = TRUE))]
# table(dt2020$origDoW, dt2020$fixedDoW) head(dt2020[origDoW != fixedDoW])
# shift to the closest aligning day 2019 Jan 1st = Tues
dt2019 <- extractDT[year == 2019]
dt2019[, `:=`(fixedDate, date2020 - 1)]
dt2019[, `:=`(fixedDoW, lubridate::wday(fixedDate, label = TRUE))]
# table(dt2019$origDoW, dt2019$fixedDoW) head(dt2019[origDoW != fixedDoW])
# 2018 Jan 1st = Mon
dt2018 <- extractDT[year == 2018]
dt2018[, `:=`(fixedDate, date2020 - 2)]
dt2018[, `:=`(fixedDoW, lubridate::wday(fixedDate, label = TRUE))]
# table(dt2018$origDoW, dt2018$fixedDoW) head(dt2018[origDoW != fixedDoW])
# 2017 Jan 1st = Sat
dt2017 <- extractDT[year == 2017]
dt2017[, `:=`(fixedDate, date2020 - 3)]
dt2017[, `:=`(fixedDoW, lubridate::wday(fixedDate, label = TRUE))]
# table(dt2017$origDoW, dt2017$fixedDoW) head(dt2017[origDoW != fixedDoW])
fixedDT <- rbind(dt2017, dt2018, dt2019, dt2020) # leve out 2016 for now
fixedDT[, `:=`(fixedDate, lubridate::as_date(fixedDate))]
fixedDT[, `:=`(fixedDoW, lubridate::wday(fixedDate, label = TRUE))]
fixedDT[, `:=`(compareYear, ifelse(year == 2020, "2020", "2017-2019"))]
# these should match table(fixedDT$origDoW, fixedDT$fixedDoW)
lastSCC <- max(fixedDT[source == "southampton.my-air.uk"]$dateTimeUTC)
diffSCC <- now() - lastSCC
lastAURN <- max(fixedDT[source == "AURN"]$dateTimeUTC)
diffAURN <- now() - lastAURN
Data for Southampton downloaded from :
Southampton City Council collects various forms of air quality data at the sites shown in Table 2.1. The data is available in raw form from http://southampton.my-air.uk.
Some of these sites feed data to AURN. The data that goes via AURN is ratifified to check for outliers and instrument/measurement error. AURN data less than six months old has not undergone this process. AURN data is (c) Crown 2020 copyright Defra via https://uk-air.defra.gov.uk, licenced under the Open Government Licence (OGL).
Latest data updated:
Table 2.1 shows the available sites and sources. Note that some of the non-AURN sites appear to have stopped updating recently. For a detailed analysis of recent missing data see Section 11.1.
t <- fixedDT[!is.na(value), .(nObs = .N, firstData = min(dateTimeUTC), latestData = max(dateTimeUTC), nMeasures = uniqueN(pollutant)),
keyby = .(site, source)]
kableExtra::kable(t, caption = "Sites, data source and number of valid observations. note that measures includes wind speed and direction in the AURN sourced data",
digits = 2) %>% kable_styling()
| site | source | nObs | firstData | latestData | nMeasures |
|---|---|---|---|---|---|
| Southampton - A33 Roadside (near docks, AURN site) | southampton.my-air.uk | 54315 | 2017-01-01 00:00:00 | 2020-04-16 06:00:00 | 2 |
| Southampton - Background (near city centre, AURN site) | southampton.my-air.uk | 146403 | 2017-01-25 11:00:00 | 2020-04-16 06:00:00 | 6 |
| Southampton - Onslow Road (near RSH) | southampton.my-air.uk | 54820 | 2017-01-01 00:00:00 | 2020-04-15 07:00:00 | 2 |
| Southampton - Victoria Road (Woolston) | southampton.my-air.uk | 40052 | 2017-01-01 00:00:00 | 2020-04-01 06:00:00 | 2 |
| Southampton A33 (via AURN) | AURN | 212298 | 2017-01-01 00:00:00 | 2020-04-15 23:00:00 | 8 |
| Southampton Centre (via AURN) | AURN | 332364 | 2017-01-01 00:00:00 | 2020-04-15 23:00:00 | 13 |
To avoid confusion and ‘double counting’, in the remainder of the analysis we replace the Southampton AURN site data with the data for the same site sourced via AURN as shown in Table 2.2. This has the disadvantage that the data is slightly less up to date (see Table 2.1).
fixedDT <- fixedDT[!(site %like% "AURN site")]
t <- fixedDT[!is.na(value), .(nObs = .N, nPollutants = uniqueN(pollutant)), keyby = .(site, source)]
kableExtra::kable(t, caption = "Sites, data source and number of valid observations", digits = 2) %>% kable_styling()
| site | source | nObs | nPollutants |
|---|---|---|---|
| Southampton - Onslow Road (near RSH) | southampton.my-air.uk | 54820 | 2 |
| Southampton - Victoria Road (Woolston) | southampton.my-air.uk | 40052 | 2 |
| Southampton A33 (via AURN) | AURN | 212298 | 8 |
| Southampton Centre (via AURN) | AURN | 332364 | 13 |
For much more detailed analysis see a longer and very messy data report.
Taken from WHO: https://www.who.int/news-room/fact-sheets/detail/ambient-(outdoor)-air-quality-and-health
yLab <- "Nitrogen Dioxide (ug/m3)"
no2dt <- fixedDT[pollutant == "no2"]
Figure 4.1 shows the most recent hourly data.
recentDT <- no2dt[obsDate > myParams$recentCutDate]
p <- makeDotPlot(recentDT, xVar = "dateTimeUTC", yVar = "value", byVar = "site", yLab = yLab)
p <- p + scale_x_datetime(date_breaks = "2 day", date_labels = "%a %d %b") + theme(axis.text.x = element_text(angle = 90,
hjust = 1))
p <- p + geom_hline(yintercept = myParams$hourlyNo2Threshold_WHO) + labs(caption = paste0(myParams$lockdownCap,
myParams$weekendCap, "\nReference line = WHO hourly threshold"))
# final plot - adds annotations
yMin <- min(recentDT$value)
yMax <- max(recentDT$value)
p <- addLockdownDateTime(p, yMin, yMax)
addWeekendsDateTime(p, yMin, yMax) + guides(colour = guide_legend(ncol = 2))
Figure 4.1: Nitrogen Dioxide levels, Southampton (hourly, recent)
Figure 4.2 shows the most recent hourly data by date and time of day.
recentDT[, `:=`(time, hms::as_hms(dateTimeUTC))]
yMin <- min(recentDT$time)
yMax <- max(recentDT$time)
p <- profileTilePlot(recentDT, yLab)
p <- addLockdownDate(p, yMin, yMax)
addWeekendsDate(p, yMin, yMax) + labs(caption = paste0(myParams$lockdownCap, myParams$weekendCap))
Figure 4.2: Nitrogen Dioxide levels, Southampton (hourly, recent)
Figure 4.3 shows the most recent mean daily values compared to previous years. We have shifted the dates for the comparison years to ensure that weekdays and weekends line up. Note that this plot shows daily means with no indications of variance. Visible differences are therefore purely indicative at this stage.
plotDT <- no2dt[fixedDate <= lubridate::today() & fixedDate >= myParams$comparePlotCut, .(mean = mean(value),
nSites = uniqueN(site)), keyby = .(fixedDate, compareYear)]
# final plot - adds annotations
yMin <- min(plotDT$mean)
yMax <- max(plotDT$mean)
p <- compareYearsPlot(plotDT, xVar = "fixedDate", colVar = "compareYear")
p <- addLockdownDate(p, yMin, yMax) + labs(caption = paste0(myParams$lockdownCap, myParams$weekendCap, myParams$noThresh))
addWeekendsDate(p, yMin, yMax) + scale_x_date(date_breaks = "7 day", date_labels = "%a %d %b", date_minor_breaks = "1 day")
Figure 4.3: Comparative Nitrogen Dioxide levels 2020 vs 2017-2019, Southampton (daily mean)
Figure 4.4 and 4.5 show the % difference between the daily means for 2020 vs 2017-2019 (reference period). In both cases we can see that NO2 levels in 2020 were generally already lower than the reference period yet are not consistently lower even during the lockdown period. The effects of covid lockdown are not clear cut…
testDate <- myParams$comparePlotCut
# testDate <- as.Date('2020-03-21')
compareYearsDiffPlot(no2dt[fixedDate >= testDate]) + labs(caption = paste0(myParams$lockdownCap, myParams$weekendCap))
Figure 4.4: Percentage difference in daily mean Nitrogen Dioxide levels 2020 vs 2017-2019, Southampton
compareYearsDiffPlotWeekly(no2dt[fixedDate >= myParams$comparePlotCut]) + labs(caption = paste0(myParams$lockdownCap))
Figure 4.5: Percentage difference in weekly mean Nitrogen Dioxide levels 2020 vs 2017-2019, Southampton
Beware seasonal trends and weather effects
yLab <- "Oxides of Nitrogen (ug/m3)"
noxdt <- fixedDT[pollutant == "nox"]
Figure 5.1 shows the most recent hourly data.
recentDT <- noxdt[!is.na(value) & obsDate > myParams$recentCutDate]
p <- makeDotPlot(recentDT, xVar = "dateTimeUTC", yVar = "value", byVar = "site", yLab = yLab)
p <- p + scale_x_datetime(date_breaks = "2 day", date_labels = "%a %d %b") + theme(axis.text.x = element_text(angle = 90,
hjust = 1)) + labs(caption = paste0(myParams$lockdownCap, myParams$weekendCap, myParams$noThresh))
# final plot - adds annotations
yMin <- min(recentDT$value)
yMax <- max(recentDT$value)
p <- addLockdownDateTime(p, yMin, yMax)
addWeekendsDateTime(p, yMin, yMax)
Figure 5.1: Oxides of nitrogen levels, Southampton (hourly, recent)
Figure 5.2 shows the most recent hourly data by date and time of day.
recentDT[, `:=`(time, hms::as_hms(dateTimeUTC))]
yMin <- min(recentDT$time)
yMax <- max(recentDT$time)
p <- profileTilePlot(recentDT, yLab)
addWeekendsDate(p, yMin, yMax) + labs(caption = paste0(myParams$lockdownCap, myParams$weekendCap))
Figure 5.2: Oxides of nitrogen levels, Southampton (hourly, recent)
Figure 5.3 shows the most recent mean daily values compared to previous years.
plotDT <- noxdt[fixedDate <= lubridate::today() & fixedDate >= myParams$comparePlotCut, .(mean = mean(value),
nSites = uniqueN(site)), keyby = .(fixedDate, compareYear)]
# final plot - adds annotations
yMin <- min(plotDT$mean)
yMax <- max(plotDT$mean)
p <- compareYearsPlot(plotDT, xVar = "fixedDate", colVar = "compareYear")
p <- addLockdownDate(p, yMin, yMax) + labs(caption = paste0(myParams$lockdownCap, myParams$weekendCap, myParams$noThresh))
addWeekendsDate(p, yMin, yMax)
Figure 5.3: Oxides of nitrogen levels, Southampton (daily mean)
Figure 5.4 and 5.5 show the % difference between the daily and weekly means for 2020 vs 2017-2019 (reference period).
compareYearsDiffPlot(noxdt[fixedDate >= myParams$comparePlotCut]) + labs(caption = paste0(myParams$lockdownCap,
myParams$weekendCap))
Figure 5.4: % difference in daily mean Oxides of Nitrogen levels 2020 vs 2017-2019, Southampton
compareYearsDiffPlotWeekly(noxdt[fixedDate >= myParams$comparePlotCut]) + labs(caption = paste0(myParams$lockdownCap))
Figure 5.5: % difference in weekly mean Oxides of Nitrogen levels 2020 vs 2017-2019, Southampton
yLab <- "Sulphour Dioxide (ug/m3)"
so2dt <- fixedDT[pollutant == "so2"]
Figure 6.1 shows the most recent hourly data.
recentDT <- so2dt[!is.na(value) & obsDate > myParams$recentCutDate]
p <- makeDotPlot(recentDT, xVar = "dateTimeUTC", yVar = "value", byVar = "site", yLab = yLab)
p <- p + scale_x_datetime(date_breaks = "2 day", date_labels = "%a %d %b") + theme(axis.text.x = element_text(angle = 90,
hjust = 1)) + labs(caption = paste0(myParams$lockdownCap, myParams$weekendCap, myParams$noThresh))
yMax <- max(recentDT$value)
yMin <- min(recentDT$value)
p <- addLockdownDateTime(p, yMin, yMax)
addWeekendsDateTime(p, yMin, yMax)
Figure 6.1: Sulphour Dioxide levels, Southampton (hourly, recent)
Figure 6.2 shows the most recent hourly data by date and time of day and time of day.
recentDT[, `:=`(time, hms::as_hms(dateTimeUTC))]
yMin <- min(recentDT$time)
yMax <- max(recentDT$time)
p <- profileTilePlot(recentDT, yLab)
addWeekendsDate(p, yMin, yMax) + labs(caption = paste0(myParams$lockdownCap, myParams$weekendCap))
Figure 6.2: Sulphour Dioxide levels, Southampton (hourly, recent)
Figure 6.3 shows the most recent mean daily values compared to previous years.
plotDT <- so2dt[fixedDate <= lubridate::today() & fixedDate >= myParams$comparePlotCut, .(mean = mean(value),
nSites = uniqueN(site)), keyby = .(fixedDate, compareYear)]
# final plot - adds annotations
yMin <- min(plotDT$mean)
yMax <- max(plotDT$mean)
p <- compareYearsPlot(plotDT, xVar = "fixedDate", colVar = "compareYear")
p <- addLockdownDate(p, yMin, yMax) + geom_hline(yintercept = myParams$dailySo2Threshold_WHO) + labs(caption = paste0(myParams$lockdownCap,
myParams$weekendCap, "\nReference line = WHO daily threshold"))
addWeekendsDate(p, yMin, yMax)
Figure 6.3: Sulphour dioxide levels, Southampton (daily mean)
Figure 6.4 and 6.5 show the % difference between the daily and weekly means for 2020 vs 2017-2019 (reference period).
compareYearsDiffPlot(so2dt[fixedDate >= myParams$comparePlotCut]) + labs(caption = paste0(myParams$lockdownCap,
myParams$weekendCap))
Figure 6.4: % difference in daily mean Sulphour Dioxide levels 2020 vs 2017-2019, Southampton
compareYearsDiffPlotWeekly(so2dt[fixedDate >= myParams$comparePlotCut]) + labs(caption = paste0(myParams$lockdownCap))
Figure 6.5: % difference in weekly mean Sulphour Dioxide levels 2020 vs 2017-2019, Southampton
Beware seasonal trends and weather effects
yLab <- "Ozone (ug/m3)"
o3dt <- fixedDT[pollutant == "o3"]
Figure 7.1 shows the most recent hourly data.
recentDT <- o3dt[!is.na(value) & obsDate > myParams$recentCutDate]
p <- makeDotPlot(recentDT, xVar = "dateTimeUTC", yVar = "value", byVar = "site", yLab = yLab)
p <- p + scale_x_datetime(date_breaks = "2 day", date_labels = "%a %d %b") + theme(axis.text.x = element_text(angle = 90,
hjust = 1)) + labs(caption = paste0(myParams$lockdownCap, myParams$weekendCap, myParams$noThresh))
yMax <- max(recentDT$value)
yMin <- min(recentDT$value)
p <- addLockdownDateTime(p, yMin, yMax)
addWeekendsDateTime(p, yMin, yMax)
Figure 7.1: 03 levels, Southampton (hourly, recent)
Figure 7.2 shows the most recent hourly data by date and time of day.
recentDT[, `:=`(time, hms::as_hms(dateTimeUTC))]
yMin <- min(recentDT$time)
yMax <- max(recentDT$time)
p <- profileTilePlot(recentDT, yLab)
addWeekendsDate(p, yMin, yMax) + labs(caption = paste0(myParams$lockdownCap, myParams$weekendCap))
Figure 7.2: Ozone levels, Southampton (hourly, recent)
Figure 7.3 shows the most recent mean daily values compared to previous years.
plotDT <- o3dt[fixedDate <= lubridate::today() & fixedDate >= myParams$comparePlotCut, .(mean = mean(value),
nSites = uniqueN(site)), keyby = .(fixedDate, compareYear)]
# final plot - adds annotations
yMin <- min(plotDT$mean)
yMax <- max(plotDT$mean)
p <- compareYearsPlot(plotDT, xVar = "fixedDate", colVar = "compareYear")
p <- addLockdownDate(p, yMin, yMax) + geom_hline(yintercept = myParams$dailyO3Threshold_WHO) + labs(caption = paste0(myParams$lockdownCap,
myParams$weekendCap, "\nReference line = WHO daily threshold"))
addWeekendsDate(p, yMin, yMax)
Figure 7.3: Ozone levels, Southampton (daily mean)
Figure 7.4 and 7.5 show the % difference between the daily and weekly means for 2020 vs 2017-2019 (reference period).
compareYearsDiffPlot(o3dt[fixedDate >= myParams$comparePlotCut]) + labs(caption = paste0(myParams$lockdownCap,
myParams$weekendCap))
Figure 7.4: % difference in daily mean Ozone levels 2020 vs 2017-2019, Southampton
compareYearsDiffPlotWeekly(no2dt[fixedDate >= myParams$comparePlotCut]) + labs(caption = paste0(myParams$lockdownCap))
Figure 7.5: % difference in weekly mean Ozone levels 2020 vs 2017-2019, Southampton
Beware seasonal trends and weather effects
yLab <- "PM 10 (ug/m3)"
pm10dt <- fixedDT[pollutant == "pm10"]
Figure 8.1 shows the most recent hourly data.
recentDT <- pm10dt[!is.na(value) & obsDate > myParams$recentCutDate]
p <- makeDotPlot(recentDT, xVar = "dateTimeUTC", yVar = "value", byVar = "site", yLab = yLab)
p <- p + scale_x_datetime(date_breaks = "2 day", date_labels = "%a %d %b") + theme(axis.text.x = element_text(angle = 90,
hjust = 1)) + labs(caption = paste0(myParams$lockdownCap, myParams$weekendCap, myParams$noThresh))
yMax <- max(recentDT$value)
yMin <- min(recentDT$value)
p <- addLockdownDateTime(p, yMin, yMax)
addWeekendsDateTime(p, yMin, yMax)
Figure 8.1: PM10 levels, Southampton (hourly, recent)
Figure 8.2 shows the most recent hourly data by date and time of day.
recentDT[, `:=`(time, hms::as_hms(dateTimeUTC))]
yMin <- min(recentDT$time)
yMax <- max(recentDT$time)
p <- profileTilePlot(recentDT, yLab)
addWeekendsDate(p, yMin, yMax) + labs(caption = paste0(myParams$lockdownCap, myParams$weekendCap))
Figure 8.2: PM10 levels, Southampton (hourly, recent)
Figure 8.3 shows the most recent mean daily values compared to previous years.
plotDT <- pm10dt[fixedDate <= lubridate::today() & fixedDate >= myParams$comparePlotCut, .(mean = mean(value),
nSites = uniqueN(site)), keyby = .(fixedDate, compareYear)]
# final plot - adds annotations
yMin <- min(plotDT$mean)
yMax <- max(plotDT$mean)
p <- compareYearsPlot(plotDT, xVar = "fixedDate", colVar = "compareYear")
p <- addLockdownDate(p, yMin, yMax) + geom_hline(yintercept = myParams$dailyPm10Threshold_WHO) + labs(caption = paste0(myParams$lockdownCap,
myParams$weekendCap, "\nReference line = WHO daily threshold"))
addWeekendsDate(p, yMin, yMax)
Figure 8.3: PM10 levels, Southampton (daily mean)
Figure 8.4 and 8.5 show the % difference between the daily and weekly means for 2020 vs 2017-2019 (reference period).
compareYearsDiffPlot(pm10dt[fixedDate >= myParams$comparePlotCut]) + labs(caption = paste0(myParams$lockdownCap,
myParams$weekendCap))
Figure 8.4: % difference in daily mean PM10 levels 2020 vs 2017-2019, Southampton
compareYearsDiffPlotWeekly(pm10dt[fixedDate >= myParams$comparePlotCut]) + labs(caption = paste0(myParams$lockdownCap))
Figure 8.5: % difference in weekly mean PM10 levels 2020 vs 2017-2019, Southampton
Beware seasonal trends and weather effects
yLab <- "PM 2.5 (ug/m3)"
pm25dt <- fixedDT[pollutant == "pm2.5"]
Figure 9.1 shows the most recent hourly data.
recentDT <- pm25dt[!is.na(value) & obsDate > myParams$recentCutDate]
p <- makeDotPlot(recentDT, xVar = "dateTimeUTC", yVar = "value", byVar = "site", yLab = yLab)
p <- p + scale_x_datetime(date_breaks = "2 day", date_labels = "%a %d %b") + theme(axis.text.x = element_text(angle = 90,
hjust = 1)) + labs(caption = paste0(myParams$lockdownCap, myParams$weekendCap, myParams$noThresh))
yMax <- max(recentDT$value)
yMin <- min(recentDT$value)
p <- addLockdownDateTime(p, yMin, yMax)
addWeekendsDateTime(p, yMin, yMax)
Figure 9.1: PM2.5 levels, Southampton (hourly, recent)
Figure 9.2 shows the most recent hourly data by date and time of day.
recentDT[, `:=`(time, hms::as_hms(dateTimeUTC))]
yMin <- min(recentDT$time)
yMax <- max(recentDT$time)
p <- profileTilePlot(recentDT, yLab)
addWeekendsDate(p, yMin, yMax) + labs(caption = paste0(myParams$lockdownCap, myParams$weekendCap))
Figure 9.2: PM2.5 levels, Southampton (hourly, recent)
Figure 9.3 shows the most recent mean daily values compared to previous years.
plotDT <- pm25dt[fixedDate <= lubridate::today() & fixedDate >= myParams$comparePlotCut, .(mean = mean(value),
nSites = uniqueN(site)), keyby = .(fixedDate, compareYear)]
# final plot - adds annotations
yMin <- min(plotDT$mean)
yMax <- max(plotDT$mean)
p <- compareYearsPlot(plotDT, xVar = "fixedDate", colVar = "compareYear")
p <- addLockdownDate(p, yMin, yMax) + geom_hline(yintercept = myParams$dailyPm2.5Threshold_WHO) + labs(caption = paste0(myParams$lockdownCap,
myParams$weekendCap, "\nReference line = WHO daily threshold"))
addWeekendsDate(p, yMin, yMax)
Figure 9.3: PM2.5 levels, Southampton (daily mean)
Figure 9.4 and 9.5 show the % difference between the daily and weekly means for 2020 vs 2017-2019 (reference period).
compareYearsDiffPlot(pm25dt[fixedDate >= myParams$comparePlotCut]) + labs(caption = paste0(myParams$lockdownCap,
myParams$weekendCap))
Figure 9.4: % difference in daily mean PM2.5 levels 2020 vs 2017-2019, Southampton
compareYearsDiffPlotWeekly(pm25dt[fixedDate >= myParams$comparePlotCut]) + labs(caption = paste0(myParams$lockdownCap))
Figure 9.5: % difference in weekly mean PM2.5 levels 2020 vs 2017-2019, Southampton
Beware seasonal trends and weather effects
Save data out for predictive models.
Save long form data to data folder.
fixedDT[, `:=`(weekDay, lubridate::wday(dateTimeUTC, label = TRUE, abbr = TRUE))]
f <- paste0(here::here(), "/data/sotonExtract2017_2020_v2.csv")
data.table::fwrite(fixedDT, f)
dkUtils::gzipIt(f)
Saved data description:
skimr::skim(aurnDT)
| Name | aurnDT |
| Number of rows | 885672 |
| Number of columns | 8 |
| _______________________ | |
| Column type frequency: | |
| character | 5 |
| Date | 1 |
| numeric | 1 |
| POSIXct | 1 |
| ________________________ | |
| Group variables | None |
Variable type: character
| skim_variable | n_missing | complete_rate | min | max | empty | n_unique | whitespace |
|---|---|---|---|---|---|---|---|
| date | 0 | 1 | 20 | 20 | 0 | 43848 | 0 |
| code | 0 | 1 | 4 | 4 | 0 | 2 | 0 |
| site | 0 | 1 | 15 | 18 | 0 | 2 | 0 |
| pollutant | 0 | 1 | 2 | 5 | 0 | 13 | 0 |
| source | 0 | 1 | 4 | 4 | 0 | 1 | 0 |
Variable type: Date
| skim_variable | n_missing | complete_rate | min | max | median | n_unique |
|---|---|---|---|---|---|---|
| obsDate | 0 | 1 | 2016-01-01 | 2020-12-31 | 2018-05-28 | 1827 |
Variable type: numeric
| skim_variable | n_missing | complete_rate | mean | sd | p0 | p25 | p50 | p75 | p100 | hist |
|---|---|---|---|---|---|---|---|---|---|---|
| value | 235811 | 0.73 | 41.61 | 74.58 | -9.6 | 3.99 | 12.2 | 37.4 | 1007.78 | ▇▁▁▁▁ |
Variable type: POSIXct
| skim_variable | n_missing | complete_rate | min | max | median | n_unique |
|---|---|---|---|---|---|---|
| dateTimeUTC | 0 | 1 | 2016-01-01 | 2020-12-31 23:00:00 | 2018-05-28 15:00:00 | 43848 |
Saved data sites by year:
t <- table(fixedDT$site, fixedDT$year)
kableExtra::kable(t, caption = "Sites available by year") %>% kable_styling()
| 2017 | 2018 | 2019 | 2020 | |
|---|---|---|---|---|
| Southampton - Onslow Road (near RSH) | 16871 | 17170 | 16507 | 4272 |
| Southampton - Victoria Road (Woolston) | 12834 | 15394 | 7694 | 4130 |
| Southampton A33 (via AURN) | 67278 | 62214 | 65751 | 17055 |
| Southampton Centre (via AURN) | 103806 | 100155 | 105871 | 22532 |
Saved pollutants by site:
t <- table(fixedDT$site, fixedDT$pollutant)
kableExtra::kable(t, caption = "Pollutants available by site") %>% kable_styling()
| no | no2 | nox | nv10 | nv2.5 | o3 | pm10 | pm2.5 | so2 | v10 | v2.5 | wd | ws | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Southampton - Onslow Road (near RSH) | 0 | 27408 | 27412 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Southampton - Victoria Road (Woolston) | 0 | 20026 | 20026 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Southampton A33 (via AURN) | 28371 | 28369 | 28371 | 23227 | 0 | 0 | 24288 | 0 | 0 | 23224 | 0 | 28224 | 28224 |
| Southampton Centre (via AURN) | 27579 | 27579 | 27579 | 21105 | 22627 | 27381 | 24908 | 26430 | 26996 | 21105 | 22627 | 28224 | 28224 |
NB:
We have also produced wind/pollution roses for these sites.
Several of these datasets suffer from missing data or have stopped updating. This is visualised below for all data for all sites from January 2020.
# dt,xvar, yvar,fillVar, yLab
tileDT <- sotonAirDT[pollutant == "no2" & dateTimeUTC > as.Date("2020-02-01") & !is.na(value)]
p <- makeTilePlot(tileDT, xVar = "dateTimeUTC", yVar = "site", fillVar = "value", yLab = yLab)
p + scale_x_datetime(date_breaks = "7 day", date_labels = "%a %d %b") + theme(axis.text.x = element_text(angle = 90,
hjust = 1))
Figure 11.1: Nitrogen Dioxide data availability and levels over time
# dt,xvar, yvar,fillVar, yLab
tileDT <- sotonAirDT[pollutant == "nox" & dateTimeUTC > as.Date("2020-02-01") & !is.na(value)]
p <- makeTilePlot(tileDT, xVar = "dateTimeUTC", yVar = "site", fillVar = "value", yLab = yLab)
p + scale_x_datetime(date_breaks = "7 day", date_labels = "%a %d %b") + theme(axis.text.x = element_text(angle = 90,
hjust = 1))
Figure 11.2: Oxides of nitrogen data availability and levels over time
# dt,xvar, yvar,fillVar, yLab
tileDT <- sotonAirDT[pollutant == "so2" & dateTimeUTC > as.Date("2020-02-01") & !is.na(value)]
p <- makeTilePlot(tileDT, xVar = "dateTimeUTC", yVar = "site", fillVar = "value", yLab = yLab)
p + scale_x_datetime(date_breaks = "7 day", date_labels = "%a %d %b") + theme(axis.text.x = element_text(angle = 90,
hjust = 1))
Figure 11.3: Sulphour Dioxide data availability and levels over time
tileDT <- sotonAirDT[pollutant == "o3" & dateTimeUTC > as.Date("2020-02-01") & !is.na(value)]
p <- makeTilePlot(tileDT, xVar = "dateTimeUTC", yVar = "site", fillVar = "value", yLab = yLab)
p + scale_x_datetime(date_breaks = "7 day", date_labels = "%a %d %b") + theme(axis.text.x = element_text(angle = 90,
hjust = 1))
Figure 11.4: Availability and level of o3 data over time
tileDT <- sotonAirDT[pollutant == "pm10" & dateTimeUTC > as.Date("2020-02-01") & !is.na(value)]
p <- makeTilePlot(tileDT, xVar = "dateTimeUTC", yVar = "site", fillVar = "value", yLab = yLab)
p + scale_x_datetime(date_breaks = "7 day", date_labels = "%a %d %b") + theme(axis.text.x = element_text(angle = 90,
hjust = 1))
Figure 11.5: Availability and level of PM 10 data over time
tileDT <- sotonAirDT[pollutant == "pm2.5" & dateTimeUTC > as.Date("2020-02-01") & !is.na(value)]
p <- makeTilePlot(tileDT, xVar = "dateTimeUTC", yVar = "site", fillVar = "value", yLab = yLab)
p + scale_x_datetime(date_breaks = "7 day", date_labels = "%a %d %b") + theme(axis.text.x = element_text(angle = 90,
hjust = 1))
Figure 11.6: Availability and level of PM 2.5 data over time
Report generated using knitr in RStudio with R version 3.6.3 (2020-02-29) running on x86_64-apple-darwin15.6.0 (Darwin Kernel Version 19.4.0: Wed Mar 4 22:28:40 PST 2020; root:xnu-6153.101.6~15/RELEASE_X86_64).
t <- proc.time() - myParams$startTime
elapsed <- t[[3]]
Analysis completed in 114.853 seconds ( 1.91 minutes).
R packages used:
Arino de la Rubia, Eduardo, Hao Zhu, Shannon Ellis, Elin Waring, and Michael Quinn. 2017. Skimr: Skimr. https://github.com/ropenscilabs/skimr.
Dowle, M, A Srinivasan, T Short, S Lianoglou with contributions from R Saporta, and E Antonyan. 2015. Data.table: Extension of Data.frame. https://CRAN.R-project.org/package=data.table.
Garnier, Simon. 2018. Viridis: Default Color Maps from ’Matplotlib’. https://CRAN.R-project.org/package=viridis.
Grolemund, Garrett, and Hadley Wickham. 2011. “Dates and Times Made Easy with lubridate.” Journal of Statistical Software 40 (3): 1–25. http://www.jstatsoft.org/v40/i03/.
Müller, Kirill. 2017. Here: A Simpler Way to Find Your Files. https://CRAN.R-project.org/package=here.
Wickham, Hadley. 2009. Ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York. http://ggplot2.org.
Zhu, Hao. 2018. KableExtra: Construct Complex Table with ’Kable’ and Pipe Syntax. https://CRAN.R-project.org/package=kableExtra.